Model Verification and Validation

In this paper we outline practical techniques and guidelines for verifying and validating simulation models. The goal of verification and validation is a model that is accurate when used to predict the performance of the real-world system that it represents, or to predict the difference in performance between two scenarios or two model configurations. The process of verifying and validating a model should also lead to improving a model’s credibility with decision makers. We provide examples of a number of typical situations where model developers may make inappropriate or inaccurate assumptions, and offer guidelines and techniques for carrying out verification and validation. 1 WHAT IS VERIFICATION AND VALIDATION? Models are used to predict or compare the future performance of a new system, a modified system, or an existing system under new conditions. When models are used for comparison purposes, the comparison is usually made to a baseline model representing an existing system, to someone’s conception of how a new or modified system will work (i.e., to a baseline design), or to current real-world system performance. In any of these cases we want to know that the model has sufficient accuracy. Sufficient accuracy means that the model can be used as a substitute for the real system for the purposes of experimentation and analysis (assuming that it were possible to experiment with the actual system). After the first phases of a modeling project – the kickoff meeting, the functional specifications document, and initial model development – the model developer enters into a series of activities called debugging, verification, model review and validation. In fact, writing and seeking approval of the functional specifications document lays the foundation for verification and validation; this is where all modeling assumptions and data requirements are laid out for the project team to review, modify if necessary and approve. In practice, a model developer intermixes debugging, verification and validation tasks and exercises with model development in a complex and iterative process. We separate them conceptually for explanatory purposes and in the hope that in practice verification and especially validation do not receive less time and effort than their due.